CMUQA in QALab-3: Essay Question Answering for University Entrance Exams

This paper describes the CMUQA entry into NTCIR QA Lab-3. CMUQA is an essay-question answering system that uses Wikipedia as its knowledge base. The essay-question portion of QA Lab-3 is composed of real exam questions from the University of Tokyo’s entrance exams which are focused on world history. The proposed system formulates answers to these questions using the following sequence of processes: question analysis, document retrieval, sentence extraction, sentence scoring, text ordering and text summarization. Despite the difficulty of this challenge and the small size of training data, CMUQA achieved the highest expert score amongst the competing end-to-end systems in the task.